Wagging: A learning approach which allows single layer perceptrons to outperform more complex learning algorithms

Despite the well known inability of single layer perceptrons (SLPs) to learn arbitrary functions, SLPs often exhibit reasonable generalization performance on many problems of interest. However, because of the limitations of SLPs very little effort has been made to improve their performance. In this paper we examine a method for improving the performance of SLPs which we call "wagging" (weight averaging). This method involves training several different SLPs on the same training data, and then averaging their weights to obtain a single SLP. We compare the performance of the wagged SLP with other more complex learning algorithms on several data sets from real world problem domains. Surprisingly, the wagged SLP has better average generalization performance than any of the other learning algorithms on the problems tested. We provide analysis and explanations for this result. The analysis includes looking at the performance characteristics of the standard delta rule training algorithm for SLPs and the correlation between training and test set scores as training progresses.